AI-Based Anomaly Detection Holds Promise in Screening for Retinal Diseases, Study Says

Anomaly detection may be useful in diabetic retinopathy screening for retinal disease and more general settings, suggest findings published in JAMA Ophthalmology. Anomaly detectors could potentially screen general populations for retinal diseases, identify novel diseases, and phenotype or detect lesser-known manifestations of common retinal diseases.

Typically, algorithms in a deep learning system (DLS), a type of artificial intelligence (AI), are trained to diagnose specific diseases based on examples in available imagery. But rare presentations of common retinal diseases or rare ophthalmic conditions have limited annotated retinal images available, making it difficult to train DLSs to recognize their respective characteristics and accurately diagnose them.

Anomaly detection, on the other hand, can potentially be trained based on only normal images of retinas to then identify any novel or abnormal presentations that may indicate disease or identify new disease phenotypes. This could be done without using images of abnormal retinas to train the algorithm. This AI could then be applied in various applications, including screening the general population, rather than used only to identify individual conditions such as diabetic retinopathy, for example.

Researchers evaluated 16 anomaly detectors using 88,692 retinal images from 44,346 individuals with diabetic retinopathy of varying degrees. Images were pulled from the publicly available EyePACS data set with fundus images labeled 0 to 4 for retinopathy severity. There are 2 categories of DLSs, both of which were included in the study. Discriminative networks classify and segment, which makes them useful in a task such as anomaly detection. Generative networks generate realistic synthetic data, like fundi showing diabetic retinopathy.

The anomaly detectors were tested with a surrogate problem: They were trained with diabetic retinopathy images showing only retinas with nonreferable diabetic retinopathy and no diabetic macular edema, no diabetic retinopathy or mild to moderate nonproliferative diabetic retinopathy. But nonreferable and referable diabetic retinopathy—including diabetic macular edema or nonproliferative diabetic retinopathy—were used to test the system’s recognition of retinal disease. Performance metrics including area under the receiver operating characteristic curve, F1 score, and accuracy were used to assess the anomaly detectors.

A combination of Deep InfoMax for embedding and a 1-class support vector machine to detect anomalies achieved an area under the receiver operating characteristic of 0.808 (95% CI, 0.789-0.827)— the best performance across all systems in the study. It was a discriminative system, 73.9% accurate, and had an F1 score of 0.755. Descriminative systems also performed best overall. In the generative category, InfoStyleGAN embedding performed the best, with a 0.667 area under the receiver operating characteristic.

Overall, the findings suggest that even with a lack of data specific to abnormalities, anomaly detection was a useful tool — particularly in the context of diabetic retinopathy screening for retinal diseases, in this case.

“When abnormal (diseased) data, i.e., referable diabetic retinopathy in this study, were not available for training of retinal diagnostic systems wherein only nonreferable diabetic retinopathy was used for training, anomaly detection techniques were useful in identifying images with and without referable diabetic retinopathy,” the authors wrote.

Limitations of the study included use of the limited surrogate problem and the availability of data only from the EyePACS system. However, study authors believe there is potential for more generalized use with further research.

“Anomaly detectors may be used to detect retinal diseases in more generalized settings and potentially could play a role in screening of populations for retinal diseases or identifying novel diseases and phenotyping or detecting unusual presentations of common retinal diseases,” they concluded.

Reference

Burlina P, Paul W, Liu TYA, Bressler NM. Detecting Anomalies in Retinal Diseases Using Generative, Discriminative, and Self-supervised Deep. JAMA Ophthalmol. Published online December 30, 2021. doi:10.1001/jamaophthalmol.2021.5557